{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "MMDAutoTrace3.00.04_en.ipynb",
      "provenance": [],
      "toc_visible": true,
      "gpuType": "T4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Welcome to MMD Auto Trace v3 for Colab! (Execution)"
      ],
      "metadata": {
        "id": "Fyy5ukQ1Snmq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Introduction\n",
        "\n",
        "The operating status and maintenance information of this tool are posted on Twitter ([@miu200521358](https://twitter.com/miu200521358)).\n",
        "\n",
        "If you get an error or it doesn't start, first check the current distribution status.\n",
        "\n",
        "I also accept inquiries by reply or DM."
      ],
      "metadata": {
        "id": "x0Du9XXeSsPi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Contents\n",
        "\n",
        "This notebook prepares and executes MMD autotracing.\n",
        "\n",
        "There is a habit of using notebooks, so please familiarize yourself with how to use \"[Preparation](https://colab.research.google.com/github/miu200521358/motion_trace_colab/blob/master/AutoTraceIntroduction.ipynb)\" .\n",
        "\n",
        "Click the \"=\" (actually three lines) at the top left of the screen. The table of contents opens. (If it is already open, proceed to the next)\n",
        "\n",
        "![目次](https://drive.google.com/uc?export=view&id=1HGk4sJmcPtMbMwcJOvE3aU1GjvKinwA_)\n",
        "\n",
        "Create an `autotrace` folder directly under Google Drive.\n",
        "\n",
        "(If it has already been created in the preparation section, it is not necessary to recreate it.)\n",
        "\n",
        "Check your notebook from top to bottom and follow the steps below one by one.\n",
        "\n",
        "Cells that need to be executed are numbered. Execute in order from ①.\n",
        "\n",
        "- **「①　Environment」**\n",
        "  - Build an environment for MMD Auto Trace v3.\n",
        "- **「②　MMD auto trace v3 execution」**\n",
        "    - Run MMD Autotrace v3.\n"
      ],
      "metadata": {
        "id": "CoaXwL-ASx6q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Errors and warnings that may occur and how to deal with them"
      ],
      "metadata": {
        "id": "f-XzbSo-TgtU"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## GPU limit exceeded"
      ],
      "metadata": {
        "id": "aEQ4dKwATlsr"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you get an error like the one below, you've reached the upper limit of GPU usage.\n",
        "\n",
        "If you are in the free area, you will be able to use it again after waiting at least 24 hours.\n",
        "\n",
        "```\n",
        "Unable to connect to GPU backend\n",
        "\n",
        "Can't connect to GPU right now because I've reached my usage limit on Colab\n",
        "If you're interested in prioritizing your GPU and increasing usage limits, check out Colab Pro.\n",
        "```\n",
        "\n",
        "![上限](https://drive.google.com/uc?export=view&id=1xxg5yM-wgNkAr1FboAM8DS2K-aqmtw5d)\n"
      ],
      "metadata": {
        "id": "Ma1e14jVToVi"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Environment crash"
      ],
      "metadata": {
        "id": "NrT6jPFBTq_x"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you get an error like the one below, your environment crashed.\n",
        "\n",
        "Delete the session once and recreate the environment\n",
        "\n",
        "```\n",
        "My session crashed after using all available RAM.\n",
        "See Colab Pro if you're interested in increased RAM runtime.\n",
        "```\n",
        "\n",
        "![クラッシュ](https://drive.google.com/uc?export=view&id=1qMoxfUggI_jDjrDifj7UgOrSmsEOMmHY)\n",
        "\n",
        "```\n",
        "The session crashed for some unknown reason.\n",
        "```\n",
        "\n",
        "![クラッシュ](https://drive.google.com/uc?export=view&id=1WFHmCzEkCLi8XnHyl7goicNQPL67RHhX)\n",
        "\n",
        "```\n",
        "Runtime > Manage sessions > Active sessions\n",
        "```\n",
        "\n",
        "![削除](https://drive.google.com/uc?export=view&id=12UaonO4UvI_HCnJI95od15_yagVIVsRD)\n",
        "\n",
        "![削除](https://drive.google.com/uc?export=view&id=1smRW97KjP8fqSy3E5dtfWJHpyBtEMKca)\n"
      ],
      "metadata": {
        "id": "TYONV48qTtSf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Automatic deletion of environments"
      ],
      "metadata": {
        "id": "n0N0LWYnTw0k"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you get the following error, the environment has already been destroyed from Colab due to the passage of time or the above crash.\n",
        "\n",
        "Delete the session once and recreate the environment\n",
        "\n",
        "![Googleドライブ連携](https://drive.google.com/uc?export=view&id=1Oa5SNwStqzR6qVMEzxg8PdLmO3FTpr1l)\n"
      ],
      "metadata": {
        "id": "ywNefafZTyy8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## GPU unused warning"
      ],
      "metadata": {
        "id": "P-eZr-2ST1bk"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The following warning may appear during the work, but there is no problem if you proceed as it is.\n",
        "\n",
        "(Because the GPU is used only for AI execution, other cells may appear while working.)\n",
        "\n",
        "```\n",
        "WARNING: You are connecting to a GPU runtime, but the GPU is not being used.\n",
        "```\n",
        "\n",
        "![警告](https://drive.google.com/uc?export=view&id=1mRW32urnPQ4LS4xrLEoPdp_XCqlq1HUF)\n"
      ],
      "metadata": {
        "id": "v28ncAfzT3db"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Remaining disk space warning"
      ],
      "metadata": {
        "id": "uH6cRFVgmfh4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you get a warning like the one below, you can safely ignore the first one.\n",
        "\n",
        "If it appears a second time, the disc may be punctured in the middle, so divide the input video at a sharp point and divide it for processing."
      ],
      "metadata": {
        "id": "cUxpejEEmlKM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "```\n",
        "Disk is low\n",
        "\n",
        "There is little free space on the disc. An error occurs when the disk is full.\n",
        "Currently using xx.xxGB out of xx.xxGB.\n",
        "You can delete files to free up disk space.\n",
        "```\n",
        "\n",
        "![容量オーバー](https://drive.google.com/uc?export=view&id=1EKt3nCK6ZYjgkNoflQNzSnW_0s4WnJYO)\n",
        "\n"
      ],
      "metadata": {
        "id": "Hae7pvkCmdps"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# terms of service"
      ],
      "metadata": {
        "id": "tUNmRfXodyvM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "If you use MMD Auto Trace or publish/distribute the results, please be sure to check the terms of use.\n",
        "\n",
        "The same is true for uses other than MMD, such as using JSON data and applying to Unity.\n",
        "\n",
        "[MMD Auto Trace v3 Terms of Use](https://github.com/miu200521358/mmd-auto-trace-3/wiki/03.%E5%88%A9%E7%94%A8%E8%A6%8F%E7%B4%84)\n",
        "\n",
        "[MMD Auto Trace v3 Technology and license used](https://github.com/miu200521358/mmd-auto-trace-3/wiki/02.%E4%BD%BF%E7%94%A8%E6%8A%80%E8%A1%93)"
      ],
      "metadata": {
        "id": "UqHhDeUAUA3V"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# ①　Environment"
      ],
      "metadata": {
        "id": "cGNYDeTFeFWW"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ①-A　Notification sound download\n",
        "\n",
        "Download the notification sound from [Sound Effect Lab](https://soundeffect-lab.info/) so that it sounds when the cell starts, completes, or fails.\n",
        "\n",
        "Please execute the cell of 【①-A】 below\n"
      ],
      "metadata": {
        "id": "c2wqFlb4eTe2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【①-A】　Notification sound download\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "from enum import Enum\n",
        "\n",
        "class SoundType(Enum):\n",
        "    START = 0\n",
        "    SUCCESS = 1\n",
        "    FAIL = 2\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "! wget --no-check-certificate -c \"https://soundeffect-lab.info/sound/anime/mp3/sceneswitch1.mp3\"\n",
        "! wget --no-check-certificate -c \"https://soundeffect-lab.info/sound/anime/mp3/incorrect1.mp3\"\n",
        "! wget --no-check-certificate -c \"https://soundeffect-lab.info/sound/anime/mp3/switch1.mp3\"\n",
        "\n",
        "from IPython.display import Audio, display\n",
        "import torch\n",
        "\n",
        "def play_sound(sound_type: SoundType, autoplay=True):\n",
        "    try:\n",
        "        if sound_type == SoundType.START:\n",
        "            file_name = \"/content/switch1.mp3\"\n",
        "        elif sound_type == SoundType.SUCCESS:\n",
        "            file_name = \"/content/sceneswitch1.mp3\"\n",
        "        else:\n",
        "            file_name = \"/content/incorrect1.mp3\"\n",
        "\n",
        "        display(Audio(file_name, autoplay=autoplay, normalize=False))\n",
        "    except:\n",
        "        print(\"■■■■■■■■■■■■■■■\")\n",
        "        print(\"■　Failed to play sound effects\")\n",
        "        print(\"■■■■■■■■■■■■■■■\")\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "exec_dict = {\n",
        "    '①-A': {'exec': False, 'premise': []},\n",
        "    '①-B': {'exec': False, 'premise': ['①-A', ]},\n",
        "    '①-C': {'exec': False, 'premise': ['①-A', '①-B', ]},\n",
        "    '①-D': {'exec': False, 'premise': ['①-A', '①-B', '①-C', ]},\n",
        "    '①-E': {'exec': False, 'premise': ['①-A', '①-B', '①-C', '①-D', ]},\n",
        "    '①-F': {'exec': False, 'premise': ['①-A', '①-B', '①-C', '①-D', '①-E', ]},\n",
        "    '②-A': {'exec': False, 'premise': ['①-A', '①-B', '①-C', '①-D', '①-E', '①-F', ]},\n",
        "    '②-B': {'exec': False, 'premise': ['①-A', '①-B', '①-C', '①-D', '①-E', '①-F', '②-A' ]},\n",
        "}\n",
        "\n",
        "NVIDIA_VISIBLE_DEVICES=all\n",
        "\n",
        "class IpyExit(SystemExit):\n",
        "    def __init__(self, text: str = None):\n",
        "        if text:\n",
        "            print(\"■■■■■■■■■■■■■■■\")\n",
        "            print(\"■　** ERROR **\")\n",
        "            print(f\"■ {text}\")\n",
        "            print(\"■■■■■■■■■■■■■■■\")\n",
        "        play_sound(SoundType.FAIL)\n",
        "        pass\n",
        "\n",
        "def check_exec_dict(cell_key: str):\n",
        "    if not torch.cuda.is_available():\n",
        "        print(\"■■■■■■■■■■■■■■■\")\n",
        "        print(\"■　** ERROR **\")\n",
        "        print(\"■ GPU is not enabled.\")\n",
        "        print(\"■ Please change the runtime to GPU while referring to the preparation section.\")\n",
        "        print(\"■■■■■■■■■■■■■■■\")\n",
        "        raise IpyExit\n",
        "\n",
        "    if not exec_dict:\n",
        "        print(\"■■■■■■■■■■■■■■■\")\n",
        "        print(\"■　** ERROR **\")\n",
        "        print(\"■ Cell 【①-A】 may not have been executed.\")\n",
        "        print(\"■ Go back from the table of contents and execute.\")\n",
        "        print(\"■■■■■■■■■■■■■■■\")\n",
        "        raise IpyExit\n",
        "\n",
        "    for p in exec_dict[cell_key]['premise']:\n",
        "        # 事前に実行されるべきセルがすべて実行されているか\n",
        "        if not exec_dict[p]['exec']:\n",
        "            print(\"■■■■■■■■■■■■■■■\")\n",
        "            print(\"■　** ERROR **\")\n",
        "            print(f\"■ Cell 【{p}】 may not have been executed.\")\n",
        "            print(\"■ Go back from the table of contents and execute.\")\n",
        "            print(\"■■■■■■■■■■■■■■■\")\n",
        "            raise IpyExit\n",
        "\n",
        "    # 全部実行されていたらOK\n",
        "    play_sound(SoundType.START)\n",
        "    return True\n",
        "\n",
        "def finish_cell(cell_key: str):\n",
        "    play_sound(SoundType.SUCCESS)\n",
        "\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** OK **\")\n",
        "    print(f\"■　Successfully executed Cell【{cell_key}】.\")\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "\n",
        "    exec_dict[cell_key]['exec'] = True\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"①-A\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "# セル終了\n",
        "\n",
        "finish_cell(ckey)\n",
        "\n",
        "print(\"\")\n",
        "print(\"\")\n",
        "print(\"■ Start cell execution\")\n",
        "play_sound(SoundType.START, autoplay=False)\n",
        "\n",
        "print(\"\")\n",
        "print(\"\")\n",
        "print(\"■ Cell execution failure\")\n",
        "play_sound(SoundType.FAIL, autoplay=False)\n",
        "\n",
        "print(\"\")\n",
        "print(\"\")\n",
        "print(\"■ Successful cell execution\")\n",
        "play_sound(SoundType.SUCCESS, autoplay=False)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "ENPQ2HkzeZdl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ①-B　Check GPU runtime\n",
        "\n",
        "Select \"Runtime\" > \"Change runtime type\" > \"GPU\" in the header.\n",
        "\n",
        "For detailed instructions, please refer to \"[MMD Auto Trace Preparation](https://colab.research.google.com/github/miu200521358/motion_trace_colab/blob/master/AutoTraceIntroduction_en.ipynb)\".\n",
        "\n",
        "If you can change it, please execute the cell of【①-B】below."
      ],
      "metadata": {
        "id": "WKqIyaMoe2wP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【①-B】　Check GPU runtime\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"①-B\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------------\n",
        "! nvidia-smi\n",
        "\n",
        "import subprocess\n",
        "try:\n",
        "    subprocess.check_output(\"nvidia-smi\", shell=True)\n",
        "except:\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** ERROR **\")\n",
        "    print(\"■　Failed to execute the nvidia-smi command.\")\n",
        "    print(\"■　① If you haven't used a GPU yet and haven't configured the runtime\")\n",
        "    print(\"■　　Set the runtime to GPU while referring to the preparation section\")\n",
        "    print(\"■　　※If you change the runtime, the environment will be reset, so please start over from 【①-A】\")\n",
        "    print(\"■　② When the usage limit of the GPU is reached and it can only be used with the CPU\")\n",
        "    print(\"■　　In the free area, you can use it again if you leave it for 24 hours or more.\")\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "# セル終了\n",
        "finish_cell(ckey)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "_dGYRo6Me7Df"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ①-C　Cooperation with Google Drive\n",
        "\n",
        "Please allow this Colab page to access your Google Drive for the following purposes:\n",
        "  - To read the trace source video saved in Google Drive\n",
        "  - To save auto trace results\n",
        "\n",
        "*Do not touch anything other than the \"autotrace\" folder in Google Drive\n",
        "\n",
        "Execute the【①-C】cell below and allow the collaboration from the subwindow.\n",
        "\n",
        "(With the latest Colab, you no longer need to copy and paste the key)"
      ],
      "metadata": {
        "id": "WfRS49WpfC0a"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【①-C】　Cooperation with Google Drive\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"①-C\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "from google.colab import drive\n",
        "import os\n",
        "\n",
        "# Googleドライブマウント\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "# 起点ディレクトリ\n",
        "base_dir_path = \"/gdrive/My Drive/autotrace\"\n",
        "\n",
        "if os.path.exists(base_dir_path):\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** OK **\")\n",
        "    print(\"■　Cooperation with autotrace folder was successful.\")\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "else:\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** ERROR **\")\n",
        "    print(\"■　I couldn't find the autotrace folder directly under Google Drive.\")\n",
        "    print(\"■　Create a autotrace folder while referring to the preparation section.\")\n",
        "    print(\"■　※If it is a spelling mistake, do not change the folder name, etc., but create a new folder.\")\n",
        "    print(\"■　　Updates after linking with Google Drive are not recognized well.\")\n",
        "    print(\"■　※Since there is a little time lag, it may not be recognized immediately after it is generated.\")\n",
        "    print(\"■　　Wait about 10 seconds and rerun the cell.\")\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    raise IpyExit\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "# セル終了\n",
        "finish_cell(ckey)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "QkEaktdxfcqE"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ①-D　Cooperation with Google SDK\n",
        "\n",
        "In order to download the data necessary for automatic tracing from miu's Google Drive, it works with `GoogleSDK`.\n",
        "\n",
        "As with cell【①-C】, give access permission from the URL."
      ],
      "metadata": {
        "id": "TAPv-jy7fz4h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【①-D】　Integration with Google SDK\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"①-D\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "# Googleドライブアクセスライブラリ\n",
        "!pip install -U -q PyDrive\n",
        "\n",
        "from pydrive.auth import GoogleAuth\n",
        "from pydrive.drive import GoogleDrive\n",
        "from google.colab import auth\n",
        "from oauth2client.client import GoogleCredentials\n",
        "\n",
        "try:\n",
        "    auth.authenticate_user()\n",
        "    gauth = GoogleAuth()\n",
        "    gauth.credentials = GoogleCredentials.get_application_default()\n",
        "    drive = GoogleDrive(gauth)\n",
        "\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** OK **\")\n",
        "    print(\"■　Successfully worked with Google SDK.\")\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "\n",
        "except Exception as e:\n",
        "    print(e.message)\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** Error **\")\n",
        "    print(\"■　Failed to link with Google SDK.\")\n",
        "    print(\"■　Please share this notebook with the author.\")\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    raise IpyExit\n",
        "\n",
        "# ----------------------------------\n",
        "\n",
        "# セル終了\n",
        "finish_cell(ckey)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "P51Ag4QigW1q"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ①-E　Code setup\n",
        "\n",
        "Build code for MMD auto-trace.\n",
        "\n",
        "Execute the cell 【①-E】 below.\n",
        "\n",
        "I get a lot of messages. It takes about 15 minutes.\n",
        "\n",
        "**Since the compilation runs on the way, there are places that seem to stop working.**\n",
        "\n",
        "**Please do not interrupt the cell by being surprised or worried. If you just wait, it will finish.**\n",
        "\n",
        "The string `ERROR` may appear, but there is a high possibility that the error is not a problem for automatic tracing, so please ignore it and proceed as it is."
      ],
      "metadata": {
        "id": "8hXO0YvKhLa_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【①-E】　Code setup\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"①-E\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "# 既存ディレクトリの削除  ---------\n",
        "! rm -r mmd-auto-trace-3\n",
        "\n",
        "# 環境変数設定  ---------\n",
        "! DEBIAN_FRONTEND=noninteractive\n",
        "! apt-get update && apt-get install -y  -o Dpkg::Options::=\"--force-confdef\" -o Dpkg::Options::=\"--force-confold\" keyboard-configuration\n",
        "\n",
        "# ライブラリの追加 ---------\n",
        "! sudo apt install libosmesa6-dev build-essential libgl1-mesa-dev libglu1-mesa-dev freeglut3-dev gcc libyaml-dev gcc\n",
        "\n",
        "# CUDA 11.7 のインストール ---------\n",
        "\n",
        "! wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin\n",
        "! sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600\n",
        "! wget https://developer.download.nvidia.com/compute/cuda/11.7.0/local_installers/cuda-repo-ubuntu2204-11-7-local_11.7.0-515.43.04-1_amd64.deb\n",
        "! sudo dpkg -i cuda-repo-ubuntu2204-11-7-local_11.7.0-515.43.04-1_amd64.deb\n",
        "! sudo cp /var/cuda-repo-ubuntu2204-11-7-local/cuda-*-keyring.gpg /usr/share/keyrings/\n",
        "! sudo apt-get update\n",
        "!apt-get -y install cuda-11-7\n",
        "\n",
        "# CUDAのPATHを変更 ---------\n",
        "\n",
        "! !ls -d /usr/local/cuda-*\n",
        "\n",
        "import os\n",
        "p = os.getenv('PATH')\n",
        "ld = os.getenv('LD_LIBRARY_PATH')\n",
        "os.environ['PATH'] = f\"/usr/local/cuda-11.7/bin:{p}\"\n",
        "os.environ['LD_LIBRARY_PATH'] = f\"/usr/local/cuda-11.7/lib64:{ld}\"\n",
        "\n",
        "# 再確認ログ ---------\n",
        "! nvcc --version\n",
        "\n",
        "# MMD自動トレースv3 clone  ---------\n",
        "version_tag = \"ver3.00.04\"\n",
        "# version_tag = \"develop\"\n",
        "\n",
        "! git clone  --recursive --depth 1 -b \"$version_tag\" \"https://github.com/miu200521358/mmd-auto-trace-3.git\"\n",
        "\n",
        "# PyTorchのインストール ---------\n",
        "! python -m pip install numpy==1.24.2\n",
        "! python -m pip install torch==1.13.0+cu117 torchvision==0.14.0+cu117 torchaudio==0.13.0 --extra-index-url https://download.pytorch.org/whl/cu117\n",
        "! python -m pip install -r /content/mmd-auto-trace-3/requirements.txt\n",
        "\n",
        "# cocoapi のインストール ---------\n",
        "import os\n",
        "import pathlib\n",
        "\n",
        "# Clone the tensorflow models repository if it doesn't already exist\n",
        "if \"models\" in pathlib.Path.cwd().parts:\n",
        "  while \"models\" in pathlib.Path.cwd().parts:\n",
        "    os.chdir('..')\n",
        "elif not pathlib.Path('models').exists():\n",
        "  !git clone --depth 1 https://github.com/tensorflow/models\n",
        "\n",
        "# Install the cocoapi\n",
        "! cd models/research/ && git clone https://github.com/cocodataset/cocoapi.git && python cocoapi/PythonAPI/setup.py build develop\n",
        "\n",
        "! cd /content/mmd-auto-trace-3/src/chumpy && python setup.py build develop\n",
        "\n",
        "# AlphaPoseのインストール ---------\n",
        "! cd /content/mmd-auto-trace-3/src/AlphaPose && python setup.py build develop\n",
        "! python -m pip install cython_bbox\n",
        "! python -m pip install -r /content/mmd-auto-trace-3/requirements2.txt\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "# セル終了\n",
        "finish_cell(ckey)"
      ],
      "metadata": {
        "id": "Wm_DfN5bhaCl",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ①-F　Data placement\n",
        "\n",
        "Place the required data in Text2Motion.\n",
        "\n",
        "Execute the cell 【①-F】 below.\n",
        "\n",
        "I get a lot of messages. It takes about 2 minutes."
      ],
      "metadata": {
        "id": "0wpzqPQ4ihVn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【①-F】　Data placement\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"①-F\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "# 3d-multi-pose-pose.zip\n",
        "downloaded = drive.CreateFile({'id': '1LtUFDGOSLW52qQwg59azrJxOfXTCtFDm'})\n",
        "downloaded.GetContentFile('/content/mmd-auto-trace-3/data/3d-multi-pose-pose.zip')\n",
        "! cd mmd-auto-trace-3/data && unzip -o ./3d-multi-pose-pose.zip\n",
        "\n",
        "# alphapose.zip\n",
        "downloaded = drive.CreateFile({'id': '16MITkRde9HVyr-6CVC55PkEYlTokwdG_'})\n",
        "downloaded.GetContentFile('/content/mmd-auto-trace-3/data/alphapose.zip')\n",
        "! cd mmd-auto-trace-3/data && unzip -o ./alphapose.zip\n",
        "\n",
        "# posetriplet.zip\n",
        "downloaded = drive.CreateFile({'id': '1Z_L0bq78sA66piDbcP3OEMPv0pImdrHU'})\n",
        "downloaded.GetContentFile('/content/mmd-auto-trace-3/data/posetriplet.zip')\n",
        "! cd mmd-auto-trace-3/data && unzip -o ./posetriplet.zip\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "# セル終了\n",
        "finish_cell(ckey)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "VGB11PgjisJ1"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# ②　MMD auto trace v3 execution"
      ],
      "metadata": {
        "id": "5ek2ZWYJkMqb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ②-A　Placement of trace source video\n",
        "\n",
        "Prepare the video file you want to trace.\n",
        "\n",
        "  - The file name should be **half-width alphanumeric characters only**. opencv cannot read double byte characters.\n",
        "  - Too long videos will run out of disk space. If it exceeds 6000f, please divide it.\n",
        "  - Place it directly under the **autotrace** folder in Google Drive.\n",
        "  - **Overwriting or updating files on the Goole drive after mounting is not recognized correctly.** New files must be uploaded with a new name before processing.\n",
        "  - Enter only the file name (including extension).\n",
        "  - After uploading the file, please execute the【②-A】cell below.\n"
      ],
      "metadata": {
        "id": "30ajJ1K3kW69"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【②-A】　Placement of trace source video\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"②-A\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "input_video_name = \"input.mp4\"  #@param {type: \"string\"}\n",
        "\n",
        "import os\n",
        "input_video_path = os.path.join(base_dir_path, input_video_name)\n",
        "\n",
        "if not os.path.exists(input_video_path):\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** ERROR **\")\n",
        "    print(\"■ The specified video was not found directly under autotrace.\")\n",
        "    print(\"■ ・ Please double check that the files are placed in the order of My Drive > [autotrace] folder > Specified image\")\n",
        "    print(\"■ ・ Recognition may not be in time if you place it immediately, so please wait about 10 seconds and try again\")\n",
        "    print(\"■　File Path: \" + input_video_path)\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    raise IpyExit\n",
        "else:\n",
        "    ! cp \"$input_video_path\" .\n",
        "\n",
        "    input_video_path = f\"/content/{input_video_name}\"\n",
        "\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "    print(\"■　** OK **\")\n",
        "    print(\"■ Placement of the trace source video is complete.\")\n",
        "    print(\"■ File Path: \" + input_video_path)\n",
        "    print(\"■■■■■■■■■■■■■■■\")\n",
        "\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "# セル終了\n",
        "finish_cell(ckey)"
      ],
      "metadata": {
        "cellView": "form",
        "id": "625x35Vaky9I"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## ②-B MMD auto trace v3 execution\n",
        "\n",
        "Execute MMD auto trace for the trace source video.\n",
        "\n",
        "It takes about 20 minutes for each minute of video.\n",
        "\n",
        "When the trace is finished, save the trace result in the `autotrace` folder in Google Drive."
      ],
      "metadata": {
        "id": "9Tinu8_mlnzN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "#@markdown 【②-B】　MMD auto trace v3 execution\n",
        "\n",
        "#@markdown ■■■■■■■■■■■■■■■■■■\n",
        "\n",
        "# セル実行可否\n",
        "ckey = \"②-B\"\n",
        "check_exec_dict(ckey)\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "# import torch\n",
        "# torch.multiprocessing.set_sharing_strategy(\"file_descriptor\")\n",
        "# torch.multiprocessing.set_start_method('spawn')\n",
        "\n",
        "import datetime\n",
        "process_img_dir_name = f\"{os.path.basename(input_video_path).replace('.', '_')}_{datetime.datetime.now():%Y%m%d_%H%M%S}\"\n",
        "process_img_dir_path = os.path.join('/content/output', process_img_dir_name)\n",
        "os.makedirs(process_img_dir_path, exist_ok=True)\n",
        "\n",
        "! cd /content/mmd-auto-trace-3/src && python executor.py --video-file \"$input_video_path\" --parent-dir \"$process_img_dir_path\" --process prepare,alphapose,multipose,posetriplet,mix,motion --verbose 20 --log-mode 0 --lang en\n",
        "\n",
        "import os\n",
        "if os.path.exists(\"/content/mmd-auto-trace-3/log/quit.log\"):\n",
        "    raise IpyExit(\"MMD automatic tracing failed due to an unexpected error.\")\n",
        "\n",
        "# 結果をGoogleドライブにコピー\n",
        "output_path = os.path.join(base_dir_path, process_img_dir_name)\n",
        "os.makedirs(output_path, exist_ok=True)\n",
        "\n",
        "import shutil\n",
        "\n",
        "# alphapose.mp4\n",
        "shutil.copy(os.path.join(process_img_dir_path, \"02_alphapose\", \"alphapose.mp4\"), os.path.join(output_path, \"alphapose.mp4\"))\n",
        "\n",
        "# mix(json)\n",
        "shutil.copytree(os.path.join(process_img_dir_path, \"05_mix\"), os.path.join(output_path, \"01_json\"))\n",
        "\n",
        "# viz_pos.html\n",
        "shutil.copy(os.path.join(\"/content/mmd-auto-trace-3/data/viz_pos.html\"), os.path.join(output_path, \"01_json\", \"viz_pos.html\"))\n",
        "\n",
        "# motion(vmd)\n",
        "shutil.copytree(os.path.join(process_img_dir_path, \"06_motion\"), os.path.join(output_path, \"02_motion\"))\n",
        "\n",
        "os.makedirs(os.path.join(output_path, \"trace_model\"))\n",
        "\n",
        "# pmx\n",
        "shutil.copy(os.path.join(\"/content/mmd-auto-trace-3/data/pmx/trace_model.pmx\"), os.path.join(output_path, \"trace_model\", \"trace_model.pmx\"))\n",
        "\n",
        "# pmx(tex)\n",
        "shutil.copytree(\"/content/mmd-auto-trace-3/data/pmx/tex\", os.path.join(output_path, \"trace_model/tex\"))\n",
        "\n",
        "# readme\n",
        "shutil.copy(os.path.join(\"/content/mmd-auto-trace-3/data/readme_ja.txt\"), os.path.join(output_path, \"readme.txt\"))\n",
        "\n",
        "# ----------------------------\n",
        "\n",
        "# セル終了\n",
        "finish_cell(ckey)"
      ],
      "metadata": {
        "id": "Q32Z-oOqlZLd",
        "cellView": "form"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}